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ABSTRACT 



This research review discusses the use of 
traditional psychometric devices to predict an individual's academic 
performance, and to determine his admittance into a particular 
curriculum, particularly in today's open-door, comprehensive, 
community- junior college. To begin with, most reported uses of the 
typical achievement or aptitude test appear to ignore such critical 
elements as: (1) factors influencing the size of the validity 

coefficient, (2) descriptors of the sample used for correlations, (3) 
the standard error of estimate, and (4) standardized regression 
coefficients (betas) in the actual regression equations. In addition, 
past research has demonstrated that little can be meaningfully added 
to high school GPA as a predictor of college success. A more 
appropriate alternative is to determine entry-level skills required 
by a particular course or curriculum, and to develop procedures that 
permit evaluation of these skills. This encourages course and 
curricular admission criteria to be related as closely as possible to 
course content, rather than to a construct called academic aptitude 
(a construct whose only operational definition is a test score). Such 
an approach can be implemented through counseling and encouraging 
students to attempt only those courses for which they have a high 
probability of success. (JO) 
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Prologue 

During the spring of 1970, an organization of junior 
college institutional research people, at their bi-monthly 
meeting, invited a speaker who is a professional tester from 
a large and prestigious testing service. His topic was en- 
trance and placement testing in the junior college. At the 
conclusion of his prepared but informal talk, I asked him 
to name one junior college, four-year college, or university 
that used entrance and placement testing programs appro- 
priately— and he was unable to do so. Though he was in- 
timately familiar with dozens of testing programs through- 
out the country, and presumably very knowledgeable about 
mental and psychological tests, he could not recommend 
a single institution of higher education that could serve as 
a model for the colleges represented at the meeting. 

This paper will first describe, in a ger era! fashion, the 
uses made of entrance and placement tests in the junior 
colleges as .sported in documents submitted to the Clear- 
inghouse. Second, it will examine in detail the psycho- 
metric model and its assumptions as reflected by junior 
colleges’ use of standardized tests. Third, it will make 
recommendations for an alternative approach to the test- 
ing programs, presently being implemented by junior 
colleges. 

Research on Testing Programs in the Junior College 
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A previous Research Review by Roueche and Boggs 
(January 1968) listed 21 Clearinghouse papers on entrance 
and placement testing. Of those listed, nine were described 
<%nd commented on (kindly) by the authors. At that time, 
almost all the research papers dealt either with the devel- 
opment of local norms for standardized tests or with the 
use of test scores in predicting academic success. For the 
present Research Review , a search of the Cleaiinghouse 
material on entrance and placement testing yielded 43 
additional papers in the same area. In reading through all 
of these, I conclude nothing has changed in the last three 
years. Junior colleges continue to develop local norms and 
use various standardized tests to attempt to predict aca- 
demic success. 

Typical of the papers developing norms are: Diablo 
Valley College (ED 010 737), which reports percentile of 
norms on SCAT; Los Angeles City College, which has been 
using the SCAT for many years and reports a consistency 
™er time in their student population on this test; the 




Virginia State Council for Higher Education, {ED 012 
183), where SAT norms were developed, used CEEB 
scores; and Daytona Beach Junior College (ED 015 725) 
has developed its own norms for both SCAT and STEP. 
Virginia and Daytona have separate norms for different 
curricula, including one set using over 200 pre-engineer- 
ing students. 

Most papers on testing submitted to the Clearinghouse 
report studies on the prediction of academic success. A 
number of these, in the Northwest particularly, examine 
the Washington Pre-College Test as a predictor (ED 012 
624, ED 017 248, ED 032 043). Others use the SAT or 
ACT scores to compute their correlations and develop 
their regression equations (ED 012 181, ED Oil 196, 
ED 029 626). In general, the results give zero order and 
multiple correlation ranging from .50 to slightly over .60. 
Though some of the studies report the actual regression 
equations, those that do so use “b-weights” rather than 
betas and seldom keep an uncontaminated holdout sample 
for cross-validation. 

A few studies using the same model introduce both 
normative data and non-cognitive test scores to help im- 
prove their prediction. The predictor variables are usually 
high school grade point average (computed in a number 
of ways), the Edwards Personal Preference Schedule (ED 
024 374), and student responses to questionnaires (ED 
013 628). 

Almost all the research reports that use high school 
grade point average as a predictor for college success find, 
as has been found for decades, that it is the best predictor. 
As most race track touts know by instinct, and psycho- 
metricians by logic, past performance is the best predictor 
of future performance. From the Clearinghouse reports 
and articles appearing in refereed journals, i am convinced 
that little can be meaningfully added to high school grade 
point average as a predictor of success in college— neither 
test scores nor personality variables. 

The Psychometric Model and Abuses Thereof 

There is one commonality m all the test studies sub- 
mitted to the Clearinghouse: none of them questions the 
appropriateness of the traditional psychometric model they 
use for education in general and the open-door comprehen- 
sive community college in particular. 



Let us assume, for the moment, that the classical psycho- 
metric model can be of some use to the junior college, that 
it can, in some mysterious way, improve the education we 
are peddling. Gross discrepancies remain between what 
the sophisticated psycho metrician would advocate and 
what is being done. Certainly, a philosophical gap exists 
between the use of test scores as admission criteria to 
particular curricula, and the statement of philosophy that 
is summarized by the phrase “open-door college/' Though 
it appears that test results are not used to deny students 
admission to a community college, it is equally evident 
that the student’s academic life is much affected by the 
scores he receives. It is on the basis of these that he is 
granted or denied permission to take specific courses or 
to enter certain certificated programs, particularly the 
para-professional curricula. As a consequence, the proba- 
bility of his continuing his formal education or completing 
a specified curriculum, is considerably influenced by the 
test scores he gets. 

It would appear from the documents submitted to the 
Clearinghouse that junior college researchers either are un- 
aware of the assumptions implicit in the psychometric 
model or choose to ignore them. For example: 

1. The basic assumption that test items are nothing 
more or less than a sample of behavior (and test-taking 
behavior at that) seems to have been lost. 

2. The concept of validity is frequently used with what 
appears to be no thought of what the test is valid for. 
Validity, it seems, is represented by the correlation co- 
efficient between the test or sub-test and grade point 
average. The particular factors that influence the size of 
the validity coefficient, or any other correlation, appear to 
be ignored. 

3. Descriptors of the sample used for correlations, such 
as range of talent or presence of extreme scores, are seldom 
reported in the studies, though they can frequently ac- 
count for potentially spurious correlations. 

4. The standard error of estimate, a much better concept 
of predictive validity, is seldom reported and, when it is, 
appears to be frequently misunderstood or misused. 

5. Actual regression equations are scarce in the research 
reports and, when they are given, they most often include 
b-weights rather than standardized regression coefficients'' 
(betas), making for difficulty in interpretation. 

6. In some reports using multiple regression techniques, 
predictor variables pick up a reverse sign weight, e.g., 
though there is a positive correlation with the criterion 
variable, there is a negative value in the equation. At a 
very high level of probability, this is attributable to a 
mathematical artifact as opposed to its being a real sup- 
pressor variable. 

7. It is also evident, from the papers that report regres- 
sion equations, that many of the predictor variables have 
such puny weight in the equation that they are “eyewash” 
as opposed to a real contribution to the existing variables. 

8. Few investigators take into account the ipsative na- 
ture of correlating any two paper- and-pencil tests. The 
idea that specific test-taking abilities will add to the size 
of a correlation seems ignored. 

9. Few investigators seem to differentiate between the 
significance of a correlation and its meaning. Some colleges 
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report hundreds of correlations and scores of t- tests with 
the same sample of less than a hundred students. They 
don’t seem to know that the significance of a correlation 
depends almost entirely on the number of cases used to 
compute it. 

10. Though most junior college investigators under- 
stand that coefficient of determination (r 2 ) represents the 
proportion of the criterion variance accounted for by the 
predictors, they lose sight of the fact that they have com- 
puted an “r,” using a sample of students as opposed to a 
“rho” of the true correlation for total population. As a 
consequence, with r = .31, they assume they are account- 
ing for ten per cent of the criterion variance. 

Summarizing the above assumptions and criticisms, I 
conclude the test research is being performed by sincere, 
dedicated people who are completely naive about the 
psychometric model and its assumptions. Further, it seems 
apparent that little can be added to high school grade point 
average that will have any meaningful, defensible use in 
determining which students should take what. 

And still college administrators continue to insist on 
getting scores. They want someone to make the decision 
about which student should be admitted to Transfer 
English and whicn to a registered nursing curriculum. 
Most important, they want these decisions to be made on 
some empirical basis, so that they are protected from the 
antagonism of the community. The person most likely to 
be given the job will be the director of research, the dean 
of admissions, the dean of students, or a committee from 
the department in question. 

The kind of “research” used to find the cutting scores 
normally compares frequency distributions of people who 
have done well and people who have been put on proba- 
tion or flunked out. The more sophisticated studies develop 
a correlation and regression equation with the grade point 
average as the dependent or criterion variable and the 
test score as the independent or predictor variable. The 
ultra-sophisticated use multiple regression and may even 
add some normative variables as potential predictors (e.g., 
age, sex, or social class) to a battery of test scores. From 
a review of the studies submitted to the Clearinghouse, it 
is evident that most educators, even those responsible for 
testing- are extremely- ignorant- of - what • affects-ccrrelations* 
and regression equations, and that they ignore the value 
and meaning of the standard error of estimate. 

The psychometric model is most useful for describing 
the status of groups. It is next most useful for predicting 
the performance of groups. It has some utility in describ- 
ing the status of individuals, but very little in predicting 
the performance of individuals — but the latter is ivhat it is 
most often used to do! 

The validity criterion most frequently used is grade 
point average and, though few studies are conducted to 
examine the reliability of this criterion, reliability is a 
limiting factor on any empirical validity coefficient. 

Some junior colleges with tens of thousands of students 
have no one on the staff capable of serving as test officer. 
Test scores have become God-given truths to which excel- 
lent, caring educators defer in their ignorance when mak- 
ing important decisions about their students. 

If a college maintains its faith in classical psychometrics 



and desires to improve its use of the testing program to 
make it less punitive and less out of step with the concept 
of the open-door college, I have some recommendations. 

First, deny access to test results to everyone (including 
the president of the college) who does not thoroughly 
understand the statistical concepts of standard error of 
measurement and standard error of estimate. 

Second, hire and train a test officer who thoroughly 
understands basic measurement concepts. He should know, 
for instance, that the only real difference between “achieve- 
ment’' and “aptitude” tests is the use made of the results. 

He should understand that contributions to the size of a 
correlation and the beta weights of a regression equation 
are made by extreme cases and range of talent. He should 
know the dangers involved in spuriously high correlations 
caused by the ipsative ncture of most correlational studies. 
These ideas should not be just learned and known by a 
good test officer, but thoroughly incorporated into his pro- 
fessional being. 

Third, the college should not accept on faith research 
studies from national test companies, even such ethical 
and competent organizations as Educational Testing Serv- 
ice and American College Testing Company. It should 
rather develop meaningful criterion variables on its own 
campus and conduct its own validity and norming research. 

An Alternative to the Standardized Test 

It is my opinion that the psychometric model, even when 
properly used, is not only useless, but actually antagonistic, 
i.e., destructive, in the educative process. It is the psycho- 
metricians who foist upon us such insidious constructs as 
the “normal curve” (any teacher who expects a normal 
distribution of achievement by his students should be 
fired outright); “discrimination coefficients” (we’re not try- 
ing to discriminate, but to teach), and “educational ex- 
pectancy^” (ignoring self-fulfilling prophecy). 

The revolution in higher education will shortly be a 
fait accompli. Courses, curricula, and departments are 
changing radically. The idiotic practice of punitive grading 
is disappearing. The concepts of recruitment and selection 
of students using grade point average and standardized 

test scores* are being rapidly' revised* Hundreds .of- colleges 

are combing the streets, pool halls, and jails of the ghetto 
and the barrio to get their share of black and Mexican 
students. Yet we still cling to our entrance tests as to a 
religious fetish. We treat our testing program as if it were 
separate and distinct from our instructional program. This 
should not be the case. The criteria for selecting students 
for courses and curricula should be related as closely as 
possible to the content of these courses rather than based 
on the irrelevancies described by the construct of academic 
aptitude that has only one operational definition, i.e., the 
test score. 

From the foregoing discussion and from the basic phi- 
losophy of the open-door college, it is evident that the use 
of cutting scores on standardized tests, even when done by 
experts, is an indefensible practice for the community col- 
lege. But what are the alternatives? 

It is apparent that, if students are permitted to enter 
any courses or any curricula they choose, the failure and 

O — * rate increase exponentially. It follows that we 



should advise, encourage, and counsel students to enter 
learning experiences for which they have a high probability 
of success. The use of standardized tests to predict success 
is notoriously poor. The use of tests permits us to account 
for 25% to 40% of the variance in overall grade point aver- 
age and considerably less in specific courses or subject 
fields. The logical alternative is to determine as explicitly 
as possible the entry skills required for a particular course 
or curriculum and to devise procedures to evaluate the 
student’s mastery of these skills. 

The omnipresent example for the comprehensive com- 
munity colleges is the placement of students in a transfer 
college English course or in one or more courses in re- 
medial or repair English. The stated purpose of the re- 
medial program is preparation for the transfer course. 
However, any college that has bothered systematically to 
appraise the success of remedial English as preparation for 
college English has found almost total failure. The normal 
criterion variable for original placement involves grades 
in high school courses or, more often, scores on standard- 
ized English tests. Yet the content of the college English 
course deals most often with essay writing and, to some 
extent, analysis of literature. What are the entry skills 
that are required to write a freshman theme? The armchair 
English teacher will immediately respond with a list of 
abilities measured by the standardized tests, i.e., grammar, 
spelling, vocabulary, mechanics, etc. It seems highly im- 
probable that a test company can pre-determine the ob- 
jectives a large number of teachers have for their students 
in any subject matter area. An alternative to the standard- 
ized test must be found, 

A great deal of effort would be required to specify the 
skills needed by a student for him to benefit from instruc- 
tion in writing compositions. However, once this were 
adequately accomplished, one could place the student in 
a learning experience that would appropriately lead him 
to the accomplishment of his objective, i.e., passing fresh- 
man English. Meanwhile, using a standardized English 
test or its ilk and arbitrarily defining cutting scores are 
antagonistic to every principle espoused by community 
college philosophy and by anyone who cares about help- 
ing people to learn things. It would be more useful simply 
to have students write sample compositions and let it go 
at that. • , . . . , . 

The alternative to standardized tests then is to analyze 
the goals of the instructional process for any course or 
curriculum and to help students achieve these goals. The 
barring of a student from a stated goal on the basis of 
irrelevant test scores (the practice of most colleges) is the 
method by which we will perpetuate the sadistic, irrele- 
vant, and pejorative institution of education that has de- 
veloped in this country. The comprehensive open-door 
community college cannot remain in the wings, merely 
following the lead of the academically respectable and 
educationally irrelevant universities. We must question 
eve/y quasi-religious belief and procedure we have adopt- 
ed and extrapolated over the decades. It we don’t, our 
community and our students will. 

M. Stephen Sheldon, Director 
UCLA/Danforth Junior College Program 
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